UNCLASSIFIED 


Approved  for  public  release;  distribution  is  unlimited. 


A  COMPARISON  OF  FUSION  ALGORITHMS  AND  METRICS  AND  THEIR  AGREEMENT 
WITH  VISUAL  PERCEPTION  LABORATORY  DETECTION  PROBABILITY  VALUES  (U) 

David  Bednarz,  Ph.D.,  Kim  Lane,  Dr.  Tom  Meitzler,  Euijung  Sohn,  Darryl  Bryk, 

Jennifer  Gillis,  Mary  Bienkowski 


Survivability  Technology  Area 


TACOM 
Warren,  MI 

meitzlet@tacom.army.mil 


ABSTRACT  (U) 


(U)  The  Visual  Perception  Laboratory  (VPL)  of  TARDEC  has  developed  a  camera  system  that 
provides  panoramic  image  fusion  that  can  be  integrated  into  various  types  of  vehicles.  The 
camera  system  is  a  combination  of  infrared  (IR)  and  visual  cameras  that  generates  a 
panoramic  image  and  provides  situational  awareness  to  the  crew  of  an  armored  vehicle  prior 
to  egress  as  well  as  hemispherical  awareness  during  patrol  activity.  Image  fusion  is  being 
used  more  and  more  by  the  US  Army  for  target  recognition  and  situational  awareness. 
Several  algorithms  for  fusing  imagery  are  tested  by  the  authors  and  the  quality  of  the  fused 
images  are  assessed  using  metrics  from  the  literature  and  compared  to  experimental  values 
obtained  in  the  Visual  Perception  Lab.  This  work  supports  a  joint  TARDEC/NVESD  STO  on 
Situational  Awareness. 

Introduction  (U) 

(U)  A  requirement  of  the  Future  Combat  System  (FCS)  and  homeland  security  is  that  armored  vehicles 
have  a  system  that  is  able  to  provide  close-in  situational  awareness  and  understanding  to  the  crew  within 
the  whole  360  degree  hemisphere  (Fig.  1)  of  the  vehicle.  TACOM,  Ford  Laboratories,  and  Samoff  Labs 
are  partnering  to  meet  this  requirement.  1 


Fig.  1:  (U)  360  degree  panoramic  image  fusion 
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System  Description  (U) 

(U)Elmo  QN42  visual  cameras  (Fig.  2)  are  used  in  the  system  because  of  their  small  size  and  excellent 
color  fidelity.  The  cameras  have  a  field  of  view  of  approximately  53  degrees  by  39  degrees.  The  Elmo 
cameras  have  a  410,000  pixel  color  CCD  that  results  in  786  (Y)  X  470  lines  (H)  resolution  with 
simultaneous  Y/C  and  composite  video  outputs. 


Fig.  2  (U)  Elmo  QN42  Camera 


(U)  Indigo  Omega  infrared  cameras  (Fig.  3)  were  also  selected  because  of  their  small  size  and  clear 
image  and  have  an  image  array  resolution  of  160  by  120  pixels,  with  a  5 1  by  5 1  micron  pixel  size.  The 
detector  is  an  uncooled  microbolometer.  The  infrared  cameras  are  fitted  with  8.5  mm  lenses  that 
provide  a  field  of  view  of  approximately  55  by  40  degrees.  The  Indigo  Omega  cameras  are  sensitive  to 
the  7.5  to  13.5  micron  band  of  the  electromagnetic  spectrum. 


Fig.  3  (U)  Indigo  Omega  IR  camera 

(U)  There  are  four  visible  cameras  and  four  IR  cameras  in  each  of  the  housings  in  the  front  and  rear  of 
the  vehicle,  hence  a  total  of  eight  visible  and  infrared  cameras.  The  output  of  the  eight  cameras  are 
combined  using  multiplexers  and  with  the  Samoff  stitching  and  image  registration  software  provide  a 
panoramic  view  that  is  scrollable  and  adjustable  in  magnification.  The  imagery  from  the  cameras  is 
combined,  registered,  and  fused  to  provide  a  real-time  panoramic  stitched  view  of  the  world  around  the 
vehicle  onto  which  they  are  mounted.  The  sensors  are  an  open  configuration  for  testing  and 
characterization.  Future  plans  for  hardening  include  the  use  of  transparent  covers  and  lenses. 
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Fused  Images:  Contrast 


Fig.  4  (U):  Fused  images  using  the  contrast  fusion  algorithm 
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Fused  Images:  Discreet  Wavelet  Transform 


Fig.  5  (U):  Fused  images  using  the  Discreet  Wavelet  Transform 
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Fused  Images:  Laplacian 


Fig.  6  (U):  Fused  images  using  the  Laplacian  fusion  algorithm 
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Fused  Images:  Morphological 


Fig.  7  (U):  Fused  images  using  the  Morphological  fusion  algorithm 


(IF)  Method 

(U)  A  total  of  46  IR  and  visible  images  taken  with  the  IR  camera  and  visible  camera  of  the  PIF  system 
were  fused  using  the  MATLAB  FuseTool  program.  Four  fusion  algorithms  were  chosen  for  comparison 
in  this  test.  See  Figures  1  through  Fig.  4  below  for  samples  of  the  images.  Subjects  were  told  to  view 
the  image  and  designate  with  a  mouse  where  a  person  was  standing.  Laboratory  probability  of  detection 
values  were  obtained  for  each  image.  Image  metrics  were  computed  for  each  image  and  compared  to 
detection  values. 
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(U)  Analysis 
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(U)  The  graph  below  in  Fig.  5  shows  a  comparison  of  the  laboratory  percent  correct  detection  rates  to 
three  metrics  that  were  used  on  each  image.  The  metrics  used  were,  1)  a  Shannon  entropy  measure,  2)  a 
textured  clutter  metric,  and  3)  a  Schmeider-Weathersby  variance  clutter  metric. 2,3 


Comparison  of  Lab  Pd  versus  Fusion  Algorithm 
(correlation  of  lab  Pd  to  entopy  =  0.9) 


Contrast  DWT  Laplacian  Morphological 

Metric 


Fig.  8  (U):  Graph  of  laboratory  detection  and  image  metric  values 

(U)  The  analysis  shows  that  the  Laplacian  and  Contrast  image  fusion  algorithms  perform  the  best.  The 
PIF  system  currently  used  in  the  Survivability  prototypes  uses  a  Laplacian  image  fusion  algorithm. 


□  PD 

■  Entropy  *  10 

□  Text.  Clut 

□  Clutter 


Results  and  Future  Work  fU) 

The  authors  have  developed  a  statistical  method  to  compare  and  rank  image  fusion  algorithms.  The 
Laplacian  and  Contrast  image  fusion  algorithms  performed  the  best  in  this  series  of  laboratory  tests.  The 
authors  pan  to  capture  more  complex  data  sets  involving  color,  more  background  clutter,  and  noise  for 
metric  and  fusion  algorithm  comparison. 
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